Rewards Analytics and Distribution Dashboard for Quantification Review¶

This document processes the outputs of the praise reward system and performs an analysis of the resulting token reward distribution.

General round summary¶

  • This period covers praise given between 2022-05-30 and 2022-06-18.
  • We allocated a total of 2814 TEC tokens for rewards.
  • Duplicate praise received a weighting of 0.1 the value of the original praise.
  • We assigned 4 quantifiers per praise instance.
  • Praise receiver names were not hidden behind pseudonyms during quantification

Praise Data Visualization¶

Since praise gets valued on a scale, we can take a look at how often each value of the scale gets assigned by quantifiers. Note: This metric disregards scores of praise marked as a duplicate, since the score of the original is already being taken into account.

Top 10 highest rated contributions¶

The ten highest rated contributions for this round were the following:

Avg. score To Reason
76.0 Juankbell#7458 for a GREAT Introduction to TE in Spanish!
72.0 Vyvy-vi#5040 for development of the praise app, including several bug fixes and improvements to the discord bot functionality and settings
55.0 Vyvy-vi#5040 for all the amazing development updates to praise! including dark mode, admin user management, bug fixes and adding multi-wallet functionality
55.0 Vyvy-vi#5040 [TRANSCRIBED MAY 5TH] for development work on the praise dashboard and the rewards system.
55.0 Nuggan#5183 for their work restructuring the RAD tool, rebuilding the development architecture and making it more moduler and better to use for future commmunities
47.0 Juankbell#7458 for preparing TEC material to represent TEC in the Intro to Token Engineering in Spanish, on June 10th. It's so good to have you excited and involved!
46.5 griff (💜, 💜)#8888 for reminding us of important directions to focus on and for putting himself available to help and advise
42.0 r-x-x#8344 for his hard work in building a unified treasury report showing all of the balances of TEC multisig wallents in one unified view
41.0 durgadas#9280 for the great effort and initiative in creating the website for GravityDAO
39.33 Tam2140#9361 for keeping the stewards on track and organized sprint after sprint

Praise Reward Distribution¶

We can now take a look at the distribution of the received praise rewards. You can toggle the inclusion of the different sources by clicking on the legend.

This is a histogram of the tec_praise_dist object. It's stored in /reward_systems/straight_distribution as a regular python module. Apart from perfoming the analysis, it can also output a visual representation with a specific header (above) and description text.

Praise Giving Distribution¶

We can also take a look at the amount of praise different users gave.

TODO Praise giver hist text

Praise Flows¶

Now for something more fun: let's surface the top "praise flows" from the data. Thanks to @inventandchill for this awesome visualization! On one side we have the top 15 praise givers separately, on the other the top 25 receivers. The people outside the selection get aggregated into the "REST FROM" and "REST TO" categories.

This graph visualizes the top 'praise flows' from 'tec_praise_dist'. Thanks to @inventandchill for his awesome work! On one side we have the top 15 praise givers separately, on the other the top 25 receivers. The people outside the selection get aggregated into the 'REST FROM' and 'REST TO categories.

Out[9]:

Quantifier Data¶

Now let's take a closer look at the quantification process and the quantifiers:

Praise Outliers¶

To aid the revision process, we highlight disagreements between quantifiers.

Outliers sort by spreads¶

This graphic visualizes controversial praise ratings by sorting them by the "spread" between the highest and lowest received score.

Please keep in mind that this is a visual aid. If there are several praise instances with similar spread and quant score, all but one end up "hidden" on the chart. For an exhaustive list, take a look at the exported file "praise_outliers.csv" .

Out[10]:
"col_dismissed = [f'DISMISSED {k+1}' for k in range(NUMBER_OF_QUANTIFIERS_PER_PRAISE)]\ncol_dupids = [f'DUPLICATE ID {k+1}' for k in range(NUMBER_OF_QUANTIFIERS_PER_PRAISE)]\ncol_scores= [f'SCORE {k+1}' for k in range(NUMBER_OF_QUANTIFIERS_PER_PRAISE)]\n\nsort_by_controversial = praise_tools.spread_sort(praise_distribution.copy(), NUMBER_OF_QUANTIFIERS_PER_PRAISE)"
Out[11]:
"# Save the outlier table in an external file for easier review\npraise_outliers_csv = sort_by_controversial.to_csv(index=False)\nwith open('praise_outliers.csv', 'w') as f:\n    f.write(praise_outliers_csv)"

TODO Sort by controversial description text.

                           ID                      DATE       TO USER ACCOUNT  \
0    6294f91390ff2413f6a07209  2022-05-30T17:04:19.117Z         durgadas#9280   
1    6294f91390ff2413f6a0720d  2022-05-30T17:04:19.499Z       acidlazzer#5796   
2    6294f91390ff2413f6a07211  2022-05-30T17:04:19.887Z   chuygarcia.eth#6692   
3    6294f91490ff2413f6a07215  2022-05-30T17:04:20.943Z             EFRA#7972   
4    629508d390ff2413f6a0721c  2022-05-30T18:11:31.282Z     eduardovegap#0422   
..                        ...                       ...                   ...   
632  62ab7747a12b884b0d8c1c02  2022-06-16T18:32:39.519Z       casale.eth#6107   
633  62ab7b65a12b884b0d8c1c0c  2022-06-16T18:50:13.363Z         GideonRo#3175   
634  62ab7c9ea12b884b0d8c1c16  2022-06-16T18:55:26.614Z  angieberryberry#9822   
635  62abb949a12b884b0d8c1c35  2022-06-16T23:14:17.683Z        natesuits#4789   
636  62ad85f7a12b884b0d8c1c4b  2022-06-18T07:59:51.939Z  divine_comedian#5493   

           TO USER ACCOUNT ID                              TO ETH ADDRESS  \
0    621f79d495a79aa51435c33f  0xfF6d6a7718A234A84f740480Dbb07f6c09cF1cD5   
1    621f79d095a79aa51435b5c8  0x8aA27E90E139d5AB5704df69429341cbCb2d2464   
2    621f79ce95a79aa51435b297  0xB22981bA3FE1De2325935c91a3B717168fB86714   
3    6234fed66cf31bb86af46e0d  0x438cDDcc8E6f0E0E18CE33F6b5a8fFCFAb5A999c   
4    621f79ce95a79aa51435b28c  0x1d921dff757610FbDB0073479e12c0a07d382677   
..                        ...                                         ...   
632  621f79de95a79aa51435ebcd  0x29FFeBCa51ecD940cb37EF91ff83cD739553b93e   
633  621f79d595a79aa51435c5ff  0xB08F0ca73E8A59a1b7970e22B8ed2F9142c3fA53   
634  621f79da95a79aa51435d52a  0xc163041D404e7657530491577665E40Da84C9Bd7   
635  621f79cf95a79aa51435b2f7  0x66dF5A7715C5D4AfbBFA52435c66aa20733be0d1   
636  621f79cf95a79aa51435b4cc  0x320c338BCF70bAAaE26e96201C33B48105Bc62C2   

             FROM USER ACCOUNT      FROM USER ACCOUNT ID  \
0              acidlazzer#5796  621f79d095a79aa51435b5c8   
1              acidlazzer#5796  621f79d095a79aa51435b5c8   
2              acidlazzer#5796  621f79d095a79aa51435b5c8   
3              acidlazzer#5796  621f79d095a79aa51435b5c8   
4                 Vyvy-vi#5040  621f79ce95a79aa51435b207   
..                         ...                       ...   
632               Tam2140#9361  621f79ce95a79aa51435b225   
633               Tam2140#9361  621f79ce95a79aa51435b225   
634  pat.zip (TE Academy)#5266  621f79d995a79aa51435d3aa   
635  pat.zip (TE Academy)#5266  621f79d995a79aa51435d3aa   
636            innov8tor3#3988  621f79d195a79aa51435b94e   

                               FROM ETH ADDRESS  \
0    0x8aA27E90E139d5AB5704df69429341cbCb2d2464   
1    0x8aA27E90E139d5AB5704df69429341cbCb2d2464   
2    0x8aA27E90E139d5AB5704df69429341cbCb2d2464   
3    0x8aA27E90E139d5AB5704df69429341cbCb2d2464   
4    0x2fA130927814B590623031e846f059df9554FdDe   
..                                          ...   
632  0xcf79C7EaEC5BDC1A9e32D099C5D6BdF67E4cF6e8   
633  0xcf79C7EaEC5BDC1A9e32D099C5D6BdF67E4cF6e8   
634  0x9F6b969Ee3F9C2b0a528b3029049e34BBD4bdB49   
635  0x9F6b969Ee3F9C2b0a528b3029049e34BBD4bdB49   
636  0xBCAD90F4a1ED4067d9bC33D4089FF18A82250253   

                                                REASON  \
0             for attending the weekly twitter sync :)   
1             for attending the weekly twitter sync :)   
2             for attending the weekly twitter sync :)   
3             for attending the weekly twitter sync :)   
4    for the awesome epic planinng board for commun...   
..                                                 ...   
632  for covering for Comms today and presenting in...   
633  bringing the Pol.is tool to the TEC. It's so m...   
634                              for her birthday!! 🎉🎉   
635  for excellent work when revising TE Academy's ...   
636  for their consistent and reliable attention to...   

                                         SOURCE ID  ... QUANTIFIER 1 USERNAME  \
0    DISCORD:810180621930070088:810180622336262195  ...       Mike Brunt#0745   
1    DISCORD:810180621930070088:810180622336262195  ...       Mike Brunt#0745   
2    DISCORD:810180621930070088:810180622336262195  ...       Mike Brunt#0745   
3    DISCORD:810180621930070088:810180622336262195  ...       Mike Brunt#0745   
4    DISCORD:810180621930070088:887255327110758400  ...       hanners717#2022   
..                                             ...  ...                   ...   
632  DISCORD:810180621930070088:810180622336262195  ...       hanners717#2022   
633  DISCORD:810180621930070088:810180622336262195  ...           zhiwei#1356   
634  DISCORD:810180621930070088:810180622336262195  ...         tigeroid#2951   
635  DISCORD:810180621930070088:810180622336262195  ...           zhiwei#1356   
636  DISCORD:810180621930070088:932649711259766865  ...         missgene#7773   

                       QUANTIFIER 1 ETH ADDRESS  QUANTIFIER 2 USERNAME  \
0    0x457Ea659aAAbD496550264c6d9d6351666f2fB4b             teaci#6108   
1    0x457Ea659aAAbD496550264c6d9d6351666f2fB4b           liviade#1387   
2    0x457Ea659aAAbD496550264c6d9d6351666f2fB4b        acidlazzer#5796   
3    0x457Ea659aAAbD496550264c6d9d6351666f2fB4b        acidlazzer#5796   
4    0x711CDABCFdEDEf2c82D9dA533268D8bB6b7468f8           liviade#1387   
..                                          ...                    ...   
632  0x711CDABCFdEDEf2c82D9dA533268D8bB6b7468f8        acidlazzer#5796   
633  0x1f83A7b86480E6142A85558Ef6999f6091041906             teaci#6108   
634  0xd3898129d4acb0F8039b1B9D6367B236F1E6100f        acidlazzer#5796   
635  0x1f83A7b86480E6142A85558Ef6999f6091041906           liviade#1387   
636  0xbE6f56Ed9E10E5F58876F7bDF1C83EeDaa721fbc        acidlazzer#5796   

                       QUANTIFIER 2 ETH ADDRESS  QUANTIFIER 3 USERNAME  \
0    0x1bE3505328880F1f204c664d430a9d2817d9D4cc      osycross(🦎🦎)#5943   
1    0xF8D1d34956cEa24718cf8687588D6FeDbc6d9AA6      osycross(🦎🦎)#5943   
2    0x8aA27E90E139d5AB5704df69429341cbCb2d2464              EFRA#7972   
3    0x8aA27E90E139d5AB5704df69429341cbCb2d2464           Tam2140#9361   
4    0xF8D1d34956cEa24718cf8687588D6FeDbc6d9AA6           Mettodo#7010   
..                                          ...                    ...   
632  0x8aA27E90E139d5AB5704df69429341cbCb2d2464      Usua◎ Silver#2618   
633  0x1bE3505328880F1f204c664d430a9d2817d9D4cc      Usua◎ Silver#2618   
634  0x8aA27E90E139d5AB5704df69429341cbCb2d2464           Mettodo#7010   
635  0xF8D1d34956cEa24718cf8687588D6FeDbc6d9AA6              EFRA#7972   
636  0x8aA27E90E139d5AB5704df69429341cbCb2d2464      Usua◎ Silver#2618   

                       QUANTIFIER 3 ETH ADDRESS QUANTIFIER 4 USERNAME  \
0    0xFe99402763531258875949D449db20157b77Ec5C         Zeptimus#3359   
1    0xFe99402763531258875949D449db20157b77Ec5C          lee0007#8152   
2    0x438cDDcc8E6f0E0E18CE33F6b5a8fFCFAb5A999c             enti#1546   
3    0xcf79C7EaEC5BDC1A9e32D099C5D6BdF67E4cF6e8         Zeptimus#3359   
4    0x7710E7e881333Fd3396Ca5966BeD6E72746b612D          Vyvy-vi#5040   
..                                          ...                   ...   
632  0x914C0E63F98a091e74b7c5e78993bB8de735aE74          Vyvy-vi#5040   
633  0x914C0E63F98a091e74b7c5e78993bB8de735aE74          lee0007#8152   
634  0x7710E7e881333Fd3396Ca5966BeD6E72746b612D  divine_comedian#5493   
635  0x438cDDcc8E6f0E0E18CE33F6b5a8fFCFAb5A999c          Vyvy-vi#5040   
636  0x914C0E63F98a091e74b7c5e78993bB8de735aE74          lee0007#8152   

                       QUANTIFIER 4 ETH ADDRESS AVG SCORE  PERCENTAGE  
0    0xaa79B87DC8B046A5E4f7D03F1562D7fe5BF98737      2.33    0.000414  
1    0x3E31155a1c17c9F85e74828447aec412090a4622      2.33    0.000414  
2    0x6897aBe0fD62B589260784B21F4ca3F78A0D8017      3.00    0.000534  
3    0xaa79B87DC8B046A5E4f7D03F1562D7fe5BF98737      3.00    0.000534  
4    0x2fA130927814B590623031e846f059df9554FdDe      7.00    0.001245  
..                                          ...       ...         ...  
632  0x2fA130927814B590623031e846f059df9554FdDe      8.00    0.001423  
633  0x3E31155a1c17c9F85e74828447aec412090a4622     10.00    0.001779  
634  0x320c338BCF70bAAaE26e96201C33B48105Bc62C2      2.33    0.000414  
635  0x2fA130927814B590623031e846f059df9554FdDe     14.50    0.002579  
636  0x3E31155a1c17c9F85e74828447aec412090a4622     11.33    0.002015  

[637 rows x 33 columns]

Praise score by quantifier -- outliers among the quantifiers?¶

Let's see how different quantifiers behaved by showing the range of praise scores they gave.

To interpret the box plot:

  • Bottom horizontal line of box plot is minimum value

  • First horizontal line of rectangle shape of box plot is First quartile or 25%

  • Second horizontal line of rectangle shape of box plot is Second quartile or 50% or median.

  • Third horizontal line of rectangle shape of box plot is third quartile or 75%

  • Top horizontal line of rectangle shape of box plot is maximum value.

Out[13]:
'quant_boxplot = quantifier_rating_table[[\'QUANT_ID\', \'QUANT_VALUE\']].copy()\nfig_box = px.box(quant_boxplot, x="QUANT_ID", y="QUANT_VALUE", points=False)\nfig_box'
Out[14]:
'\npr = praise_tools.praise_quantifier(praisedata=praise_distribution,quantifiertable=quantifier_rating_table)\n'

Score displacement: tendency to under/over-scoring?¶

Out[15]:
'fig=pr.plot_mean_displacement()\nfig.show()'

Scoring correlation: how similiar am I scoring with others?¶

Out[16]:
'fig=pr.plot_coefficient()\nfig.show()'

Agreement on duplication¶

Out[17]:
"# pre-process to remove None\ndup_agree_check = sort_by_controversial.copy()\n\nfor k in range(1,NUMBER_OF_QUANTIFIERS_PER_PRAISE+1):\n    col = 'DUPLICATE ID '+str(k)\n    dup_agree_check.loc[dup_agree_check[col].isnull(),col]=0\n\ndup_agree_check['DUPLICATION AGREED'] = [len(set(kk))==1 for kk in dup_agree_check[col_dupids].values]"
Out[18]:
"# output the duplication disagreement\nduplication_disagreement = dup_agree_check.loc[dup_agree_check['DUPLICATION AGREED']!=True,:]\n\nduplication_disagreement = duplication_disagreement.drop(['SPREAD']+col_scores+col_dismissed, axis=1)\n\nmd(f'Among **{len(dup_agree_check)}** praises, **{len(duplication_disagreement)} ({len(duplication_disagreement)/len(praise_distribution)*100:.2f}%)** do not agree on duplication')"

Praise instances with disagreements in duplication are collected in 'results/duplication_examination.csv'. To compare, look at the last 4 columns: 'DUPLICATE MSG 1/2/3' and 'ORIGINAL MSG'.

Out[19]:
"duplication_disagreement['ORIGINAL MSG']=duplication_disagreement['REASON'] # replicate this column just after the other messages for easy comparison\n\nduplication_disagreement.to_csv('duplication_examination.csv')"

Agreement on dismissal¶

Out[20]:
"# pre-process to remove None\ndism_agree_check = sort_by_controversial.copy()\n\nfor k in range(1,NUMBER_OF_QUANTIFIERS_PER_PRAISE+1):\n    col = 'DISMISSED '+str(k)\n    dism_agree_check.loc[dism_agree_check[col].isnull(),col]=0\n\n\ndism_agree_check['DISMISSAL AGREED'] = [len(set(kk))==1 for kk in dism_agree_check[col_dismissed].values]\n\ndismiss_disagreement = dism_agree_check.loc[dism_agree_check['DISMISSAL AGREED']==False,:]\ndismiss_disagreement= dismiss_disagreement.drop(col_scores+col_dupids+['SPREAD'],axis=1)"
Out[21]:
"dismiss_disagreement['ORIGINAL MSG']=dismiss_disagreement['REASON'] # replicate this column just after the other messages for easy comparison\n\ndismiss_disagreement.to_csv('dismissal_disagreed.csv')\n\nmd(f'Among **{len(dism_agree_check)}** praises, **{len(dismiss_disagreement)} ({len(dismiss_disagreement)/len(praise_distribution)*100:.2f}%)** do not agree on dismissal')"

Praise instances with disagreements in dismissal are collected in'results/dismissal_disaggreed.csv'. You can further look into who dismissed and who did not.

Save the distribution files¶